E-14 F Reward Evaluation and Customization F-19 F.1 Load Shifting Penalty ( LS F-19 F.2 Default Reward Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-19 F.3 Customization of Reward Formulations . . . . . . . . . . . . . . . . . . . . . . . Current Workload - The current workload level, which includes both flexible and non-flexible components. The data center modeled is illustrated in Figure 1. The hot air exits the cabinets and returns to the CRAH via the ceiling.

Designing system algorithms remains challenging, where the discontinuous nature of the solution space often forces system engineers to rely on generic heuristics at the expense of performance. We study whether LLMs can practically drive algorithm generation, and find that they are biased towards well-known generic designs, rather than making the creative leaps needed to navigate the discontinuous solution space. To address this limitation, we introduce MetaMuse, a framework for creative ideation built on three self-reflection principles: (1) quantifying solution diversity and usefulness in measurable performance space, rather than abstract idea space, (2) steering ideation through external stimuli, rather than internal randomness, and (3) constructing executable solutions using waypoint reasoning, rather than free-form chain-of-thought. Extensive evaluation shows that MetaMuse can generate high-performing solutions for two critical problems at a global cloud provider: cache replacement (reducing cache misses by up to 35.76%) and online bin packing (reducing bin usage by up to 30.93%).

This is the accepted author's version of the paper. The final published version is available in the Proceedings of the 2016 I EEE International Conferences on Ubiquitous Intelligence and Computing (UIC), Advanced and Trusted Computing (ATC), Scalable Compu ting and Communications (ScalCom), Cloud and Big Data Computing (CBDCom), Internet of People (IoP), and Smart World Congress (SmartWorld). Distributed and Intelligent Systems Research Group University of Westminster London, United Kingdom Leszek.Sliwko@my.westminster.ac.uk Distributed and Intelligent Systems Rese arch Group University of Westminster London, United Kingdom V.S.Getov@ westminster.ac.uk Abstract -- This paper presents the Accurate Google Cloud Simulator (AGOCS) - a novel high - fidelity Cloud workload simulator based on parsing real workload traces, whic h can be conveniently used on a desktop machine for day - to - day research. Our simulation is based on real - world workload traces from a Google Cluster with 12.5K nodes, over a period of a calendar month. The framework is able to reveal very precise and detai led parameters of the executed jobs, tasks and nodes as well as to provide actual resource usage statistics. The system has been implemented in Scala language with focus on parallel execution and an easy - to - extend design concept. The paper presents the det ailed structural framework for AGOCS and discusses our main design decisions, whilst also suggesting alternative and possibly performance enhancing future approaches. The framework is available via the Open Source GitHub repository. Correctly characterizing user behavior is of utmost importance when modeling Cloud workloads [14, 19] . Cloud workloads have been researched thoughtfully and are relativ ely wel l understood [15, 16, 24, 27]; however there have been limited attempts to accurately simulate Cloud workloads with consideration of detailed t ask parameters and constraints [6, 7, 10, 13], especially with consideration of workload scheduling [25]. Evaluat ing the performance of distributed applications and services without unrestricted access to existing Cloud environments is a very difficult task, which can also be addressed via simulation. Existing Cloud simulators do succeed in representing high - view inf rastructure parameters (i.e.

This research investigates how Machine Learning (ML) algorithms can assist in workload allocation strategies by detecting tasks with node affinity operators (referred to as constraint operators), which constrain their execution to a limited number of nodes. Using real-world Google Cluster Data (GCD) workload traces and the AGOCS framework, the study extracts node attributes and task constraints, then analyses them to identify suitable node-task pairings. It focuses on tasks that can be executed on either a single node or fewer than a thousand out of 12.5k nodes in the analysed GCD cluster. Task constraint operators are compacted, pre-processed with one-hot encoding, and used as features in a training dataset. Various ML classifiers, including Artificial Neural Networks, K-Nearest Neighbours, Decision Trees, Naive Bayes, Ridge Regression, Adaptive Boosting, and Bagging, are fine-tuned and assessed for accuracy and F1-scores. The final ensemble voting classifier model achieved 98% accuracy and a 1.5-1.8% misclassification rate for tasks with a single suitable node.

Machine learning has driven an exponential increase in computational demand, leading to massive data centers that consume significant amounts of energy and contribute to climate change. This makes sustainable data center control a priority. In this paper, we introduce SustainDC, a set of Python environments for benchmarking multi-agent reinforcement learning (MARL) algorithms for data centers (DC). SustainDC supports custom DC configurations and tasks such as workload scheduling, cooling optimization, and auxiliary battery management, with multiple agents managing these operations while accounting for the effects of each other. We evaluate various MARL algorithms on SustainDC, showing their performance across diverse DC designs, locations, weather conditions, grid carbon intensity, and workload requirements. Our results highlight significant opportunities for improvement of data center operations using MARL algorithms. Given the increasing use of DC due to AI, SustainDC provides a crucial platform for the development and benchmarking of advanced algorithms essential for achieving sustainable computing and addressing other heterogeneous real-world challenges.

In the field of high-performance computing (HPC), there has been recent exploration into the use of deep reinforcement learning for cluster scheduling (DRL scheduling), which has demonstrated promising outcomes. However, a significant challenge arises from the lack of interpretability in deep neural networks (DNN), rendering them as black-box models to system managers. This lack of model interpretability hinders the practical deployment of DRL scheduling. In this work, we present a framework called IRL (Interpretable Reinforcement Learning) to address the issue of interpretability of DRL scheduling. The core idea is to interpret DNN (i.e., the DRL policy) as a decision tree by utilizing imitation learning. Unlike DNN, decision tree models are non-parametric and easily comprehensible to humans. To extract an effective and efficient decision tree, IRL incorporates the Dataset Aggregation (DAgger) algorithm and introduces the notion of critical state to prune the derived decision tree. Through trace-based experiments, we demonstrate that IRL is capable of converting a black-box DNN policy into an interpretable rulebased decision tree while maintaining comparable scheduling performance. Additionally, IRL can contribute to the setting of rewards in DRL scheduling.

Computer systems such as storage systems normally require transparent white-box algorithms that are interpretable for human experts. In this work, we propose a learning-aided heuristic design method, which automatically generates human-readable strategies from Deep Reinforcement Learning (DRL) agents. This method benefits from the power of deep learning but avoids the shortcoming of its black-box property. Besides the white-box advantage, experiments in our storage productions resource allocation scenario also show that this solution outperforms the systems default settings and the elaborately handcrafted strategy by human experts.